Metastability in decelerating rotation of superfluid 3He-B

The maximum and the equilibrium numbers of vortex lines are measured in a rotating cylindrical container as a function of its rotation velocity X. We find that a slow cool-down through the superfluid transition at constant rotation produces a state with the equilibrium number of lines, N %2 (X). In decelerating rotation, when vortex lines annihilate, the state is obtained which includes the maximum possible number of lines, N .!9 (X). At low velocities N .!9 (X) exceeds N %2 (X) and the surplus N .!9 !N %2 is stabilized by an annihilation energy barrier, which during further deceleration ultimately vanishes in an instability. At high velocities extrinsic effects, such as a misalignment between the container and rotation axes, remove the annihilation barrier and cause N .!9 to equal N %2 . Our analysis concludes that the lowest annihilation barrier is provided by the bending of the peripheral vortex lines at one end of the container and not by collective instability in the peripheral shape of the vortex array. ( 1998 Elsevier Science B.V. All rights reserved. PACS: 67.57.Fg; 47.32.!y; 05.70.Fh